DOCUMENT RESUME 



ED 051 834 



LI 002 80 4 



AUTHOR 

TITLE 



INSTITUTION 

SPONS AGENCY 
PUB DATE 
NOTE 



Moriarty, John H. ; Seibert, Warren F. 

A Suggested Methodology for Relating Research 
Collection Criteria to Disciplines and to 
Educational Program Quality, 

Purdue Univ. , Lafayette, Ind. Audio-Visual Center; 
Purdue Univ*, Lafayette, Ind. University Libraries. 
Council on Library Resources, Inc*, Washington, D.C. 
Dec 68 

20 p* 



EDRS PRICE EDRS Price MF-$0.b5 HC-S3.29 

DESCRIPTORS ^Library Planning, ^Library Research, Library 

Services, Reference Materials, ^Research Libraries, 
Research Met hod clog y 



ABSTRACT 



The growing pressures and the complex conditions 
which now bear upon research libraries are not adequately reflected 
in the development of policies, plans or current service activities. 
In the place of current, reliable, and extensive information which 
might be reflected in decisions and actions, librarians seem to 
operate normally on the basis of only a few guidelines and these 
typically are rather outdated, arbitrary, and indiscriminate, 
especially in the way they seek to serve the various special fields 
represented in research. Improvement in the adequacy of library plans 
and decisions could be expected by systematically accumulating and 
disseminating greater quantities of new and descriptive information 
concerned with the broader environment of research libraries. 
(Author/MM) 




003 8-?y ED051834 



U.S. DEPARTMENT OF HEALTH. EOUCATION 
& WELFARE 

OFFICE OF EOUCATION 
THIS DOCUMENT HAS BEEN REPRODUCED 
EXACTLY AS RECEIVED FROM THE PERSON OR 
ORGANIZATION ORIGINATING IT POINTS OF 
VIEW OR OPINIONS STATED DO NOT NECES 
SARILY REPRESENT OFFICIAL OFFICE OF EOU 
CATION POSITION OR POLICY 



A SUGGESTED METHODOLOGY FOR RELATING RESEARCH COLLECTION 
CRITERIA TO DISCIPLINES AND TO EDUCATIONAL 
PROGRAM QUALITY 



A Planning Study Report to 
Council on Library Resources, Inc., Washington, D.C. 

by 

John H. Moriarty and Warren F. Seibert 
Libraries -Audio Visual Center 
Purdue University 



Lafayette, Indiana 
December, 1968 




Introduction 



The proposed investigation the main elements of which 
are developed here has its origins in several recent events 
confirming; the need for data helping to characterize research 
libraries and their ^oles. To begin the development, it is 
first necessary to describe several of the more troublesome 
and unresolved questions from library funders, professional 
librarians, and library users associated with research li- 
braries, since it is out of these questions about these 
libraries that we can justify the proposed work. 

It is quite apparent that research or research and 
development activities are placing constantly increasing 
burdens on a similarly growing number of research libraries. 
Professional librarians have been slow to demand or initiate 
adequate research and development on library problems, but 
there is reason enough for such initiative even in the 
morally Platonic demand that we librarians should know 
ourselves; there are also bound to be utilitarian and 
Benthamite benefits in pay-off by way of attraction and 
justification of improved funds and library services (Munn, 
1968 ). 1 In the United States and, in varying degrees, else- 
where, the indexes of activity in research and scholarship are 
advancing: funds for research are being regularly enlarged 

(see, e.g. , U.S. Bureau of Census, 1958 ), enrollments in 
graduate programs are still growing (e.g., National Center 
for Educational Statistics, 1966 ), new research specialties 
are finding favor with funding agencies and are taking their 



^Author names and dates appearing in parentheses refer 
to items in the bibliography. 
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place alongside the established and more affluent specialties 
(e.g., Carter, 1965; Keeney, 1965), and the supporting re- 
search libraries are also enlarging at an accelerating pace 
(Dunn, Seibert, 6 Scheuneman, 1968, p. 13). These are only 
some of the signs that research and attendant scholarship 
are broad and robust enterprises . 

The varieties of pressures and conplications that arise 
from these expensive , growing enterprises are almost in- 
describable and they become especially severe within research 
libraries, where the available and pertinent guidelines for 
change are meagre and too often obsolete. Even in a more 
stable setting, one with slower rates of growth and expansion, 
the difficulties would be impressive. For example, there 
still would be major problems stemming from inflation, from 
the keen competition for capable personnel, and from the 
need to exploit many new and potentially valuable technol- 
ogies. But beyond these problems, research librarians must 
try also to deal with the unpredictable ways in which special 
research fields are bom out of mergers and out of differ- 
entiation of existing fields (and often, it would seem, out 
of something akin to spontaneous generation). There are 
further strains which arise from the need to serve many 
heterogeneous specialties or clienteles — humanist and 
scientist, novice and mature investigator, wealthy and 
impecunious projects, and locally experienced or transient, 
occasional clients. 

If anyone ever seriously proposed that a research 
library's problems could be solved by the application of funds 
sufficient to allow the collection of all materials in the 
fields of interest, the time for such proposals has long since 
passed. Not only do new research publications in most 
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special fields exceed the quantity which the most efficient 
reader- specialist can survey (Licklider, 1966, p. 1044; 

Price, 1956), they also exceed by a wide margin the research 
library’s ability and resources in collecting. Both the 
reader-specialist and the library must be highly selective in 
determining which literature to consider. Furthermore, like 
the other sources of current problems in research! libraries , 
this need for greater selectivity continues to grow more 
acute , 

The growing pressures and the complex conditions which 
now bear upon research libraries are not adequately reflected 
in the development of policies, plans or currerc service 
activities of these libraries. Instead, in the place of 
current, reliable, and extensive information which might be 
reflected in decisions and actions, librarians seem to operate 
normally on the basis of only a few guidelines (e.g. , Downs, 
1967, Clapp and Jordan, 1965, Trueswell, 1966) and these 
typically are rather outdated, arbitrary, and indiscriminate, 
especially in the way they seek to serve the various special 
fields represented in research. 

There are several imnediate and clear possibilities for 
proceeding beyond the present conditions and the first of 
these might properly be to accumulate systematically and to 
disseminate greater quantities of new and descriptive infor- 
mation concerned with the broader environment of research 
libraries. From the resulting awareness of status and trends, 
some improvement in the adequacy of library plans and de- 
cisions could be expected. In addition and more specifically, 
however, for a library to respond well to the demands placed 
upon it by research and scholarship, it probably can do no 
better than to imitate the salient features of its most 
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successful counterparts. Currently, though, two principal 
obstacles effectively restrict such imitation and they are, 
first, that the detailed features of successful libraries and 
collections are essentially unknown and, secondly, that no 
important effort has been invested to identify features which 
are salient and contributive to success and others which are 
probably only correlated and coincidental. Further effort 
also needs to be given to the study of library usage differ- 
ences across fields of research or disciplines and to the 
clearer identification of usage patterns which differentiate 
special research fields. As of now, too little is known about 
these habits and their attendant special requirements , even 
though it is well known that different disciplines and 
special fields frequently have their own forms of communi- 
cation and of library materials utilization (Fenzel, 1966, 
Hemer, 1958), 

It also seems likely that an important need in research 
library planning and policy making is not just for better and 
more detailed reports, to reveal present status of research 
libraries in cross-section or as "snapshots," but also for 
greater attention to developmental, longitudinal, and dynamic 
characteristics of libraries. Quite possibly, for example , 
the number of periodical volumes in a collection is not so 
sensitive an index of quality or utility as would be an index 
of the current rate of periodicals acquisitions or the rate 
of change in these acquisitions during the past few years. 

Such suspicions stem from the knowledge that, in the sciences 
at least, a disproportionate burden is carried by a few of the 
most recent periodical volumes (Cole, 1963). The many other 
volumes, no matter how numerous, are of much less consequence 
in their contributions to that area of scholarship. 
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A desire to refine the present broad understandings of 
the features contributing to adequacy and excellence in 
research library collections comes readily from the real- 
ization that so little has yet been systematically attempted 
on these questions;, it comes also from recognizing the great 
importance the questions carry, both in their influence on 
the effectiveness of research and in their great financial 
implications. At the same time, it is understandable that 
so little work has yet been done. Many of the pressures, 
notably the heightened tempo of scholarship in action, and 
of the related questions are of rather recent origin. In 
addition, investigators are properly reluctant to undertake 
the unravelling of a complex in which causes, correlates, 
and effects are thoroughly intertwined, especially when the 
most influential variables could prove to be effectively 
beyond control — for example, perhaps "great faculties cause 
great collections." And finally all this is complicated by 
the need to refine, by means of a detailed examination, the 
library needs and uses of the various disciplines . Un- 
fortunately, "discipline" is itself one of those conven- 
iently flexible terms which can delude the user into the 
supposition of clear and specific meaning. 

Now, to review briefly, what is sought here is the 
translation of general and well-intentioned interests in the 
most central problems of research libraries into the evo- 
lution from those interests of a promising plan for the 
execution of certain selected investigations. It seems 
clear at this point that the already large responsibilities 
of these libraries are still expanding rapidly. The prob- 
lems to be dealt with cannot be met just with funds and with 
very great expansions of collections, even if these options 
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were open. Neither can plans and policies be based on the 
few, meagre and often obsolete guidelines that are used now. 
Instead, just for one thing it is time to undertake some 
empirical but also systematic work, discipline by discipline, 
to study the characteristics of collections, of their uses, 
and of the correlates and contributing causes of their 
"success-in-use . " 



Considerations in Study Design 

Intermediate between the development and description of 
any one scholarship's needs and the later statement of study 
procedures appropriate to these needs, it is desirable to 
indicate something of the evolution of the choices. The 
procedures do not of course emerge directly out of the 
described needs, but must instead be designed and selected 
to represent each need efficiently. It must be stressed also 
that with problems-needs which are as broad and as uncharted 
as those underlying the presently proposed research, the 
procedural alternatives and choices are unusually numerous ; 
beneath each of the apparent choices, there are several 
embryonic procedures and several discarded opticus. The major 
and remaining choices are discussed below. 

The focus of the contemplated research is upon the 
features of research libraries and, especially, of research 
collections which correlate with (and are most predictive of) 
success and "non-success" of the collections in their support 
of research; and scholarship. This research focus is derived 
from the inescapable conviction that current guidelines are 
weakened principally by the absence of known relationships 
between their main recommendations and separate measures of 
collections' success-in-use. To refine these important and 
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expensive decisions, they must become related to suitable 
criteria of success, to valid and procedurally independent 
information . 

The problems of locating or developing ’'suitable 
criteria. . could be acute if it were not for the recently 
completed "Cartter report’ 1 (Cartter, 1966), which fortunately 
provides an excellent basis for such criteria. As unlikely 
as it may be that institutional success in research, scholar- 
ship, and related activities can be reliably identified, it is 
nevertheless true that Cartter’s procedures and major findings 
are difficult to fault. He provides a well executed and 
generally well accepted assessment of graduate programs and 
faculties which can serve very usefully in the further study 
of many related library collections. 

Beyond the need to relate features of research collections 
to the success-in-use of these collections, there is a second 
major need also to be represented in the study procedures. It 
is that of examining systematically and in relation to suit- 
able criteria, interdisciplinary differences in the salient 
features of research collections. Again, to contrast this 
with most prevalent guidelines, they give little or no recog- 
nition to the substantial and documented differences among 
disciplines in their library service needs (see, e % g,, 

Hemer, 1958), With the continuing differentiation of dis- 
ciplines or special fields which is so evident and with the 
related differentiation of their library requirements, it is 
crucial to understand better the extent and implications of 
these differences. 

In developing the present plans and report, probably the 
most intriguing and troublesome question was that of the 
variation in the breadth or scope of disciplines. As 
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indicated earlier, the term 1 discipline" is deceptive and it 
can suggest, as perhaps it did to the promulgators of some 
existing guidelines , thar it applies with constant meaning to 
each academic specialty. French, electrical engineering, 
sociology, and physics are all disciplines and all typically 
have parallel administrative units in the organization of one 
university or another. Also, within each unit, a range of 
more specialized instructional and research programs are 
commonly conducted. However, when the task is to identify the 
quantity and varieties of literature required to support 
research in each field, are similarities still evident? In ■ 
the absence of sound information supporting the assumed 
similarity, it must be seriously questioned. No known forces 
assure that disciplines shall be created equal and their sub- 
sequent advances in academe are also influenced unevenly by 
countless forces which must serve to compound earlier dis- 
similarities. 

Proceeding on the belief that disciplines differ sub- 
stantially in scope, breadth, or intellectual variety and that 
they differ correspondingly in the quantity and varieties of 
literature required to support research, how shall these 
differences be recognized and dealt with? TWo means which 
were briefly considered and then discarded were (a) to limit 
consideration to a few disciplines for which the argument of 
equivalence could be effectively advanced, i.e., to create 
homogeneity by means of careful selection, or (b) to assume 
that breadth or scope is reflected in the size of a dis- 
cipline’s research agenda. This idea still holds some 
attraction, but its main difficulty is that disciplines rarely 
proclaim agenda and no reliable methods can be conceived for 
gaining access to their implicit agenda. 
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The method finally adopted and adapted for gauging dis- 
cipline breadth is derived from the 11 type-token ratio” , an 
index first developed and used in readability research., as well 
as in other linguistic research (see, e.g., Vogel S Washburne, 
1927-28, Carroll, 1964, Miller, 1951). Normally, this ratio is 
employed as a Treasure of verbal or vocabulary diversity (and 
indirectly, of intellectual depth or power) and as such it can 
index a characteristic of a sneaker or author analogous to 
that characteristic of a discipline which is of interest here. 
Normally, the ratio is based on the vocabulary found in a 
recorded language sample, but the present adaptation will 
substitute bibliographic entries from a sampling of a dis- 
cipline's research reports for the vocabulary elements. Never- 
theless, in either case, the point can be made that large 
ratios identify individuals or disciplines which are in- 
tellectually broad, rich, and varied. 

In a typical application of the type-token ratio, each 
word in the recorded language sample is counted as a token and 
each different (or unique) word is a type. The greater the 
number of distinct types within a given number of tokens, the 
greater the ratio. When used in the study of written 
passages, rather than the study of authors of passages, these 
ratios are found to be negatively correlated with and highly 
predictive of the comprehensibility or readability of the 
passages. Again, in an analogous way, it seems likely that 
the type-token ratio of a discipline will be at least crudely 
indicative of the extent to which people in general will 
comprehend or be conversant with the discipline . The 
' narrower ,! f less diverse disciplines which exhibit the smaller 
ratios are predicted to be the better known and better under- 
stood by laymen. (In discussing this with others, some reser- 
vations and competing predictions have emerged, including the 
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view that narrow disciplines are inevitably very specialized 
and incomprehensible to laymen. That may be, but as we 
originally surmised, narrow disciplines may be the ones which 
less reading will presumably "cover". Hence, the probability 
that a layman could have ' covered” it is greater. In any 
event , the original prediction is left standing to serve as 
a "goad”, if not as more than that.) 

In using type-token ratios, it must be recognized that 
the ration size is not independent of the size of the sample 
of elements studied. The smaller the sample, the greater is 
the likelihood that each additional element will be unique, 
thus increasing the ratio. To control this, the simplest 
procedure is probably to adopt a standard sample size for 
the purposes of type-token calculations (e.g., 100 entries) 
and if more than the standard number are available, to draw 
only the standard number from the larger group randomly. 
Somewhat related to this, it is foreseen that literature 
needs for research will not be simply and successfully pre- 
dicted just by the observed diversity in the literature which 
earlier research cites. Eoth the quantity and the diversity 
of citations should contribute to prediction, so methods for 
recognizing both deserve consideration. Finally, it is useful 
to consider further variations on the type-token ratio and 
their uses in the present context or in related work. Even 
though the proposed variation, based on bibliographic entries 
as elements, seems most promising for present purposes, other 
variations might be considered, such as ratios which treat 
authors (or only senior authors) as elements, ratios with 
periodical titles (or all "source" titles) as elements, and 
even ratios which treat the words (or morphemes) in a sample 
of research report titles as elements. Each of these 
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variations and others could bear study as means of indexing 
the scope or intellectual dispersion of research fields. 



Procedures 

Tile contemplated procedure: .■» the study may be divided 

into two categories of data gathering activity, with these 
then followed by the further procedures of data analysis and 
summarization. The two data gathering categories consist, 
first, of the procedures required to obtain quantified 
descriptions of specified research collections and, second, 
of research publications prepared by users of the specified 
collections . 

It is helpful also to conceive of the contemplated pro- 
cedures as organized into the six structural levels which are 
represented schematically in Figure 1. There, it can be seen 
that this proposed study of research collections is concerned 
with each of five common and broad fields or families of 
disciplines. These five are widely recognized but come most 
directly from Cartter (1966) and they include the humanities, 
social sciences, biological sciences, physical sciences, and 
engineering. Then, at the next level in the structure and 
within each broad field, four separate and generally repre- 
sentative disciplines are tentatively identified. In the case 
of the physical sciences, for example, the four are physics, 
chemistry, mathematics, and geology. (Before implementing 
the study, it will be desirable to review again the composition 
of each of these groups of tentatively designated disciplines 
and perhaps to substitute some other disciplines which would 
contribute to the more representative and efficient coverage 
of each broad field. For purposes of the present descriptions, 
however , such review poses no complications . ) 
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Figure i. Schematic Representation of the Relationships Among Study Procedures. 
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At the third level in the study’s structure and still 
following practices based on Cartter (1966), there is a 
division into two graduate program quality levels. The '’high 1 ' 
level includes graduate programs (thus, indirectly, specific 
universities and faculties) which Cartter identifies as 
"extremely attractive" , but in those cases where fewer than 
four programs are so classified, additional programs will be 
drawn from among those in the next lower rank. The second 
program quality level can be euphemistically dubbed as the 
"other" level and it corresponds to programs from Cartter’ s 

o 

"acceptable plus" rank. 

The fourth level in the structure is the level at which 
individual graduate programs and, thus, individual universities 
and research collections are identified. The decision has 
been made that at this level and within each quality level and 
each designated discipline, four graduate programs will be 
selected from among those identified by Cartter. The actual 
selection of individual programs might be conceived of as a 
simple matter of recording institutional names from those in 
Cartter’ s lists*, however, some complications must be expected. 
The first consideration in selection is of course that the 
graduate program add to the representativeness of the small 



2 Some thought was given to the selection and use here of 
graduate programs not mentioned or represented in the Cartter 
study, rather than those he identifies as "acceptable plus"; 
however, the stronger reasons seem to favor the present 
decision. Hie "unmentioned" programs could too readily appear 
as weak in numerous respects, including the research collections 
at their disposal and they would thus stand in stark and rela- 
tively useless contrast to the "extremely attractive' programs. 
On the other hand, the "acceptable plus" programs are them- 
selves worthy of emulation and they offer the further chal- 
lenge of distinguishing between two similar and yet separable 
complexes. By analogy, the decision was to undertake the 
discrimination of aqua from green, rather than green from red. 
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group in its class which are to be studied. But the decision 
to include must also give weight to availability of crucial 
information, to institutional cooperation, and some matters of 
efficiency in execution of the study. In short:, then, this 
fourth level encompasses the identification of four graduate 
programs (and their universities and collections) at each of 
the two quality levels within each of four disciplines which 
are within each of five broad fields. Programs will first be 
selected as representative, with this reviewed and perhaps 
modified to reflect practical considerations. 

The fifth and sixth levels in the structure can best be 
described jointly, since the one is virtually a continuation 
of the other. The fifth level simply indicates that the in- 
formation (the data) from each selected graduate program will 
pertain either to the collection used in support of the pro- 
gram's research or to the products of the research, the pub- 
lications issuing from the program. Since both collections 
and research "products" are to be described, different 
varieties of data or information are required and it is these 
varieties which are indicated in the sixth level. 

In selecting and developing the specific items of infor- 
mation which will be used to describe each collection, there 
were two principal considerations: First, as indicated 

earlier, efforts should be made to index developmental, long- 
itudinal, and dynamic features of collection, not just those 
which reflect a cross-section of static indexes. Secondly, 
it was desired that a broad range of features be included in 
the indexes. From these two principal considerations and some 
others, the following list of collection features emerged: 

1. Number of volumes 

2. Number of periodical volumes 
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3. Number of periodical subscriptions 

4. Proportion of periodicals less than 5 years old 

5. Proportion of periodicals 5-10 years old 

6. Proportion of total volumes that is periodicals 

7. Expenditures in each of the last 10 years for books, 

periodicals, and binding 

8. Proportion of the books that are less than 5 years 

old 

9. Units of microform 

10. Number and types of special ancillary formats 

11. Number of graduate students served 

12. Number of faculty served 

13. Number of FIE librarians assigned 

14. Volumes per graduate student 

15 . Volumes per faculty member 

16. Subscriptions per graduate student 

17. Subscriptions per faculty member 

18. Number of other graduate programs which depend on 

the collection. 

19. Per cent, growth in acquisition expenditures during 

the last 5 years 

20. Estimated proportion of the library burden that is 

borne by other libraries-collections . 



The above, then, are the principal items of information 
to be gathered with respect to each program and collection 
studied. The majority of these items can be readily and re- 
liably gathered, but some will of course require approximate 
procedures to be applied, including some methods of sampling 
from the collection and estimation on the basis of samples. 

To describe the research publications which issue from 
programs under study, the likely procedures consist first of 
identifying five senior faculty within each program and, for 
these purposes, it seems that ’’senior" should mean that a 
faculty member shall have been at the institution for a 
minimum of five years and have the rank of professor or 
associate professor. Then, a further group consisting of 
recent advanced degree recipients from the given program will 
be identified; these should probably be persons for whom the 
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five senior faculty were committee chairmen. For those ten 
people, their research publications for the past five years 
will be identified and a sampling of these publications will 
be drawn. A total of approximately twenty publications, 
distributed among the ten investigators, seems desirable. 
Bibliographies of the twenty publications will be recorded in 
full, Primarily to permit calculation of the applicable type- 
token ratio, but also to determine the number and spread of 
references per publication, the extent of the reliance on 
periodicals and other classes of materials , and the age of 
the referenced materials. 

The principal analyses will consist of statistically 
summarizing and contrasting the indexes derived for the broad 
fields , the disciplines , and the program quality levels and 
derived from the study of collections or research publications ' 
bibliographies. Hie statistical procedures to be used will 
be those that are commonly employed in behavioral science re- 
search — comparisons of means , nonparametric comparisons of 
frequencies, correlation analysis, and perhaps discriminant 
function analysis. In each analysis, however, the purpose will 
be to identify consistent differences and consistent patterns 
of differences which distinguish the collections associated 
with each broad field, each discipline, and each program 
quality level. A particular effort will also be made to de- 
termine the potential usefullness of the proposed type-token 
ratios in the estimation of collection requirements. If the 
ratio exhibits some of the sensitivity anticipated for it, 
further development will be warranted to refine it as an 
index useful in planning and library evaluation. In addition, 
the thousands of bibliographic entries taken from the publi- 
cations in each discipline studied will be analyzed for their 
potential use as evaluative lists. The main emphasis and 
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purpose throughout the analyses will be to translate the 
available evidence into clearly visible relationships useful 
in the guidance of research library planning and development. 



Conclusion 

There is a great distance which librarians and, es- 
pecially, research library planners need somehow to travel, 
that is, the distance between the arbitrary, sometimes very 
dubious standards which are still much in evidence and, at 
the other extreme, the full comprehension of each component 
contributing to a research library T s usefulness and value. 

That distance cannot be travelled quickly and the full dis- 
tance surely cannot be travelled at all* however, beginnings 
must be made. This proposed study is a beginning. It under- 
takes to look directly and very objectively at the real world 
and real events of research collections, their disciplines, 
and research accomplishments. From this, it hopes to achieve 
some refinement of the present understanding of these broad 
elements and their relationships. 

Eventually, given some success in these early efforts 
and given also the opportunity to refine and develop further 
these first successes, useful improvements in the current 
understandings and current planning procedures can be expected. 
In view of the great and growing importance of research and 
in view also of the crucial roles of libraries in research, 
all improvements, even small ones, are significant and welcome. 
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